Derivation of evolutionary payoffs from observable behavior. 
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Interpretation of animal behavior, especially as cooperative or selfish, is a challenge for evolution- 
ary theory. Strategy of a competition should follow from corresponding Darwinian payoffs for the 
available behavioral options. The payoffs and decision making processes, however, are difficult to 
observe and quantify. Here we present a general method for the derivation of evolutionary payoffs 
from observable statistics of interactions. The method is applied to combat of male bowl and doily 
spiders, to predator inspection by sticklebacks and to territorial defense by lions, demonstrating 
animal behavior as a new type of game theoretical equilibrium. Games animals play may be de- 
rived unequivocally from their observable behavior, the reconstruction, however, can be subjected 
to fundamental limitations due to our inability to observe all information exchange mechanisms 
(communication) . 
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Animals in the wild or laboratory environment are in- 
volved in coordinated, game-like interactions. It can be 
a fight for mating opportunities with basic individual 
choices to attack or retreat, or cooperation vs. selfish- 
ness dilemma such as participation in territory defence 
or predator inspection. The corresponding Darwinian 
rewards and costs are significant, exacting even the life 
of a player. Animal behavior in these contests, therefore, 
must be affected by natural selection, converging with 
time to some evolutionary optimum. 

Modeling of animal behavior in the framework of evo- 
lutionary game theory [U [5J requires a-priori assump- 
tions of available behavioral options and of a decision 
making mechanism, followed by subsequent analysis of 
evolutionary optimal behavior. For instance, a combat 
for mating opportunities or territorial possession can be 
modeled by the War of Attrition game [U [5]: competitors 
decide whether to retreat or pursue the fighting under 
time increasing cost for competition. An alternative de- 
scription is a strategy choice game, such as Hawk-Dove, 
with options for selfish (Hawk) and cooperative (Dove) 
behaviors. Evolutionary payoffs for possible outcomes of 
the contest (retreat vs. pursue. Hawk vs. Dove etc.) 
define evolutionarily stable strategies (ESS), capable of 
outperforming any other behavior. In the case of War of 
Attrition, for instance, evolutionarily stable strategy is to 
pursue fighting with a probability exponentially decaying 
with time. The notion of evolutionarily stable behavior 
stems from game theoretical Nash equilibrium [B], though 
may significantly deviate from it in interactions contain- 
ing significant information exchange [7 . 

Predictions of animal behavior by its evolutionary sta- 
bility are, in some cases, ambiguous and paradoxical. 
There is a longstanding argument concerning emergence 
and maintenance of cooperation or even altruism, which 
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are apparently in discord with the Darwinian ban to in- 
crease fitness of others 13 E]. For instance, contrary to the 
evolutionarily stable strategy for War of Attrition, non- 
exponential statistics of combat durations takes place in 
some species. The models, therefore, lack the ability to 
account for the variety of behavioral strategies in nature. 

Back in 1983, S. Austad proposed the combat of male 
bowl and doily spider (Frontinella pyramitela) as a quan- 
titative test of evolutionary game theory [TU]. In nature, 
mature males of this species wander in search of female 
webs with eggs to fertilize and determine their evolution- 
ary success by fighting with other males. The contests 
end when one of the combatants gains access to the eggs 
while his opponent either retreats or dies. The statistics 
of fight outcomes (percentage of fatal injuries) together 
with expected evolutionary payoffs (average eggs per fe- 
male) were accurately documented. 

One can expect a self-consistent evolutionary theory 
predicting spiders' behavior in a given fight from the ex- 
pected number of eggs per female and from lifetime male 
reproductive success: mortal combat becomes more rea- 
sonable if a future evolutionary gain is unlikely. In lab- 
oratory conditions, two virgin spiders of the same size 
with no previous knowledge of females' value, finish their 
fight in 67% of cases by death of one of the competitors. 
The surviving looser gets less than S = 5% of eggs. Ac- 
cording to field data, average value of a female is Vp = 10 
eggs and the male's lifetime reproductive success in the 
absence of fighting costs is Vl = 16.2 eggs. 

The main impediment to derive the spider's observ- 
able behavior from the corresponding evolutionary pay- 
offs is the lack of a general theory for biological cogni- 
tive processes. Assumption of random choice by a spider 
whether to retreat or continue the fight leads to a fal- 
lacious conclusion that sometimes they do not fight at 
all. Hypothesis of war of attrition (fight until one of the 
competitors retreats) predicts much shorter fight dura- 
tions than observed[T^. It indicates that even natural 
seeming assumptions, such as the skill to recognize a re- 
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treating competitor, may be beyond the spiders abilities 
and have to be incorporated in evolutionary models with 
great caution. 

In this work, the complex process of choosing either 
selfish S (e.g. continue the fight) or cooperative C (e.g. 
retreat) behavior is described by conditional probabilities 
a and /?, where a and 1 — a are respectively the proba- 
bilities to choose cooperation C and selfish S behaviors 
against the competitor's unconditional selfish behavior S, 
while P and 1 — /3 are the probabilities to choose cooper- 
ation C and selfish S behaviors against the competitor's 
unconditional cooperation behavior C. In this context, 
a and (3 can be viewed as selfishness aversion and co- 
operation attraction respectively. These two parameters 
suffice to describe all four types of interactions: S vs. S, 
S vs. C, C vs. S and C vs. C. This approach gener- 
alizes previous works based on conditional description of 
behavior [HI [121 1131 [HI [HI and eliminates any assump- 
tions about specific decision making mechanism. 

The conditional probabilities a and /? can be derived 
experimentally in two ways. A first, bottom-up ap- 
proach, is to determine {aobserv^ Pobserv) from observable 
field data ^pq, the statistics of outcomes p vs. q in the 
course of a competition (p, q G {S, C}) (see Fig. 1). The 
conditional probability aobserv to choose cooperation C 
against the competitor's unconditional selfish behavior S, 
is the ratio between the unconditional probability ^cs 
(to be cooperative vs. selfish competitor) and the un- 
conditional probability of the competitor to be selfish 
O-cs + ^SS- Applying the same reasoning for Pobserv one 
obtains: 
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A second, top-down, approach derives the evolution- 
arily stable behavior {acaic, f^caic) from observable payoff 
values Wpq, for p vs. q interaction: 
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This approach permits to calculate the acaic and Pcaic 
parameters only in cases where no symmetrical coopera- 
tion rice = or symmetrical selfishness Uss — exist 
(see Methods). For the case ^Icc — one obtains: 
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No prediction based solely on statistics of interactions 
can be made for other cases. To be self-consistent, any 
evolutionary explanation of behavior has to achieve a rea- 
sonable match between the values {ctobservT Pobserv) and 

{^calc: locale)- 



In the case of the bowl and doily spider, the observed 
interaction statistics are figs = 0.67, ^cc = 0, ^sc = 
^cs = 0.165, assuming that only selfish vs. selfish (67%) 
interactions lead to death of one of the combatants. Fol- 
lowing eq. ([T]), the corresponding observed conditional 
probabilities are Uobserv — 0.198 and Pobserv = 0. The 
derived payoff table from Vp = 10, = 16.2 and 5 = 5% 
is (see Methods): 
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Following eq. ([3|, the corresponding calculated condi- 
tional probabilities are acaic = 0.286 and (3caic = and 
flss = 0.55, flee = 0, flsc = flcs = 0.225 (see eq. 
(12l). An exact match between {aobserv, f^observ) and 
{acaic, Pcaic) IS achieved if the lifetime gain value is low- 
ered to Vl — 13.42. This correction seems reasonable 
since the original estimation of Vl = 16.2 did not take 
into account naturally existing threats such as predators, 
etc [10]. 

The analysis of this work is sufficient to derive a self 
consistent link between evolutionary payoffs Wpq and be- 
havior characteristics which depend solely on outcomes 
statistics ^pq. For instance, mutual information ex- 
change between competitors / is such a characteristic 1 1 5 j . 
with a calculated value of / = 0.047 bits (out of a maxi- 
mum of 1 bit describing a binary interaction) (see Meth- 
ods, eq. (18l). It indicates a low level of communication 
between fighting males. Only the link between payoffs 
and f2pg depending phenomena may be universal. Other 
observable parameters, such as duration of fight [ini [H], 
can not be predicted in the framework of this work and 
are in general species' specific. 

One can derive the payoff table Wpq (pi from observ- 
able statistics of interaction outcomes slpg, even when 
direct measurement of the payoffs is impossible. When 
the observed interaction statistics lack symmetrical co- 
operation or selfish interactions {il.cc — or Uss — 0) 
then the values of corresponding payoffs follow from eqs. 
^ and ffl respectively, using {aobserv, l3observ) ^ in- 
stead of (ctcaic, Pcaic)- But for cascs where ilcc and 
^ss 7^ 0, the reconstruction of payoffs from statistics is 
impossible (see Methods). 

Evolutionarily stable behavior {a^aic, Pcaic) follows 
from known payoffs Q using eqs. (D and ^. The 
corresponding results are presented in Fig. 2, where pay- 
offs are separated in standard games [HI IH] of Leader 
(6 > c, c > 1), Battle of the Sexes {b < c,b > 1), Chicken 
(6 > 1, < c < 1) and Prisoner's dilemma (6 > 1, c < 0). 
Other evolutionarily stable behaviors corresponding to 
the same payoffs may exist, since a general stability anal- 
ysis is impossible for behaviors with either /3 or a 1 
(see Methods). The value /3 = 0, however, is the most 
intuitive behavior for c < b, since it suggests always a 
selfish behavior versus a cooperative competitor (if c > 6 
then a = 1 is the complementary case) . One can hypoth- 
esize, therefore, that the derived behavior {acaic, Pcaic) is 
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stable and unique in a majority of real world interactions. 

Applying this reasoning to the problem of predator in- 
spection by stickleback fish [12], we can argue that stick- 
lebacks do not play the games of Chicken, Leader or Bat- 
tle of the Sexes, since the observed ilcc = 0.207, ilss = 
0.621 rise = ^cs = 0.086 (leading to {aotserv = 0.121 



and Pobserv — 0.708), see eq. ([Ij)) are not negligible (see 
Supplementary Material (SM)). It supports indirectly the 
conclusions that Prisoner's dilemma is the most probable 
type of the corresponding interaction and highlights the 
importance of communication in the course of the preda- 
tor inspection process [S]. Mutual information exchange 
of sticklebacks eq. ([Ts]) is J = 0.24 bits, five times greater 
than that of the bowl and doily spiders. According to the 
work, additional information concerning processes of mu- 
tual communication must be taken into account to derive 
a corresponding payoff table. 

The developed method allows to analyze the main 
properties of behavior even in the case when exact statis- 
tics of interaction are unknown. For instance, in the case 
of territory defense by female lions[20' specific individuals 
(leaders) cooperatively invest in territory defense, while 
laggards selfishly avoid risk of fighting. The individual 
role develops early in life and persists into adulthood. 
A leader cooperatively takes a significant personal risk 
even when she recognizes the lagging behavior of others. 
On a speculative level, one can suggest that the uncon- 
ditional cooperation stems from development of evolu- 
tionarily stable behavior characterized by solitary acts of 
cooperation flcs,^SC 7^ 0, with subsequent degradation 
of the ability to change roles. Such development requires 
payoffs corresponding to Chicken, Leader or Battle of the 
Sexes games. 

From a game theoretical perspective, the evolution- 
arily stable behavior conditions ([3| and Q correspond 
to a new type of game equilibrium. Whereas the Nash 
equilibrium]^ does not consider the possibility of com- 
munication between players (thus taking into account 
only uncorrelated responses (mixed strategies)), and 
correlated |21j /communication E5] equilibria consider 
only fixed information exchange mechanisms, our ap- 
proach assumes that information exchange mechanisms 
and behaviors co-evolve, eventually reaching together a 
stable state. It applies to evolving interactions where the 
competitors observe each other and exchange information 
mutually [3]. 

To conclude, evolutionary payoffs are demonstrated to 
follow unequivocally from observable statistics of interac- 
tions lacking symmetrical cooperation or selfishness. The 
results are verified by using known payoffs and interac- 
tion statistics of male combat in bowl and doily spider. 
In addition, the impossibility to derive payoffs from ob- 
servations is demonstrated in the case of coexisting acts 
of mutual cooperation and selfishness. These results may 
explain solitary acts of cooperation as in the case of terri- 
torial defense by lions and support Prisoner's dilemma as 
the main model for predator inspection by sticklebacks. 
The correspondence of the derived and observed behav- 



iors supports the possible existence of a unified game the- 
oretical framework for animal behavior [7]. 



I. METHODS 

A. Evolutionarily stable points 

To derive eqs. ^ and one should find for each pay- 
off table Wpq an evolutionarily stable behavior {ast, /?«*), 
that outperforms any other behavior (mutant) to, char- 
acterized by {am,Pm)- 
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where G{ai, l3i\aj,Pj) is the gain of an individual (a^, /3i) 
interacting with {aj,j3.j). On the boundary of the (a,/3) 
space (a = 0, a — 1, (i — Q or (3 ~ 1), the stable values 
{oLgt,j3st) correspond to the maxima of the gain in the 
tangential direction to the boundary: 
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where Xi denotes either or (3i (e.g. if /? = then the 
tangential direction Xi is ai). Mutants along perpendic- 
ular direction do not affect the values {ast,Pst) though 
may make the corresponding behavior to be unstable (see 
SM, Fig. 3). 

The gain G{ai, (3i\aj, (3j) is determined by the proba- 
bilities flpq of an interaction p vs. q, and by the corre- 
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sponding payoffs Wpq, 

Gia,,P,\aj,Pj) = ncc + cncs + bnsc. (9) 

taking eq. ([2| into account. 

In the course of an interaction between individuals 
{ai, l3i) and {aj,l3j), the probabilities Q.pq are determined 
by a system of equations analogous to eq. ([T])): 
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This system, comprising 5 equations for 4 variables, is 
solvable for five distinct cases only: if the competitors are 
identical and on the boundaries of (a, /3) space. The cor- 
responding solution for identical competitors (0^,/?^) = 
{aj,f3j) = {a,p) is: 

«^ o _ (l-«)(l^/3) 
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Second, if they are on the /3 = boundary {f3i = Pj = 0): 

(1 - a,){l - aj) 



^cc — 0' — 



ss 
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(12) 



Thkd, if they are on the a = I boundary (ai — aj — 1): 
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Fourth, {ui = a J = 0): fiss = 1, ^sc = ^^cs = 
0. Fifth, (A = Pj = 1): f^cc = 1, ^^5C = f^cs 
0. 

Eqs. ^ and Q for {ast,Pst) follow from eqs. Q 
and ([9]) using (12 1 and (13) respectively. This derivation 
considerers the mutants along the boundaries only (see 
SM, Fig. 3). No specific {ast,Pst) exists in the fourth 
and fifth cases (a = and P = 1 boundaries), since the 
corresponding flpq are independent of a and p. 

A general stability analysis of particular behavior 
(a,/3) is impossible. It requires analysis of host vs. mu- 
tant interactions that can not be described by conditional 
probabilities (a, P). For instance, when mutants leave the 
boundary, the system of eqs. (10) is unsolvable. In case 



of the bowl and doily spider the stability of behavior at 
(a = 0.286, P = 0) along the P axis is justified mainly by 
observation (see SM, Fig. 3). Stability of behavior along 
a axis can be justified only by simulation of population 
dynamics (see SM), since the stability condition ([8| van- 
ishes for the gain ([9) in the cases described by statistics 
(12) and (13), with behaviors ^ and ^ respectively. 



and 

Consequently, the parameters b and c in ([2| are: 

^ Wsc - Wss 
Wcc-Wss' 
^ ^ Wcs - Wss 
Wcc - Wss ' 



(16) 
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These transformation do no affect the stability conditions 
^ andj8| (see SM). 

Eq. (jS^foUows substituting Vp — 10, Vl — 16.2 and 
6 = 5% to eqs. plland (fT7|. 



C. Information calculation 

Mutual information per interaction is given bv|15j: 

-(1 _ Q,)(l-")7(1 _ /J)(1-/3)(1-7)q,"7^/3(1-7) 
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where: 
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For spiders / = 0.047 bits, substituting (a = 
0.198, /3 = 0) in eq. (18). The same result for stickle- 
backs is / = 0.24 bits, (a = 0.121, /3 = 0.708). 



II. SUPPLEMENTARY MATERIALS 



A. Solution of system ( 10 1 



B. Reduction of payoff table Wpq to two parameter 
b and c form 



The payoffs of biological organisms competing for a 
resource with value Vp, and possessing lifetime gain Vl 
are: 
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where each of them receives Vf/2 in case of mutual co- 
operation, Vf/2 — Vl/2 for mutual selfishness (assuming 
that it leads to the death of one of the combatants) , while 
VpS and Vf(1 — 5) are the shares of interacting selfish and 
cooperative competitors respectively. 

Reduction of the payoff table ( 14 1 to its two parameters 
form ([2| requires two transformations: 
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The system of eqs. ( 10 ) can be represented as 
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A solution for this system exists if the determinant of the 
corresponding extended matrix U vanishes: 
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For a general case there is no solution, since it includes 
five equations and only four variables. 
The determinant of U : 



aiiPi - 1)(32 + a2{aiP2 - Pi{ai + P2 - !))• 



(23) 



vanishes either in case of identical phenotypes (ai, (3i) — 
{ct2,P2) or at the boundaries of the (a, /3) space, a = 0, 1 
or /3 = 0, 1. It can be demonstrated as follows: 



det U{a, P, a, (3) = 0, 



Then taking into account (24l, a2 7^ ai fits (21 1 if: 

vanishes. The same holds for changes in j32- 
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The solutions (111, (12 1 and (13 1 can be checked by a 
substitution. 



dynamics is described by the replicator dynamics equa- 
tions 124 : 
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where p{ai,Pi) the density of the behavior {ai,Pi), Tg^n 
is the time span of a single generation and F(ai,f3i) = 
J2j Pio^jj /3j)G{ai, /3i\aj, Pj) is the evolutionary fitness 
(average gain) of the phenotype {ai,Pi). The average 
fitness F in the population is F = ^ ■ p{ai, Pi)F{ai, Pi). 

In the case of bowl and doily spiders, convergence of a 
population to a = 0.286 (corresponding to condition ([3| 
in case of payoffs (6 — 1.556, c = 0.444)) is checked by 
a simulation of replicator dynamics eq. ( |29| taking into 
account gain ([9|, statistics (12 1 and prediction ^ (see 
the attached animation). The same convergence holds 
for all solutions ^ with payoffs {b, c) corresponding to 
games of Chicken (6>l,0<c<l) and Leader (6 > 
c, c> 1). 

The stability of P values corresponding to condition 
^ can be verified for the entire region of Battle of the 
Sexes game {b < c,b > 1) using gain ([9| with statistics 
p|. 



D. 



Calculation of flpq, a and /3 in case of predator 
inspection by sticklebacks 



Normalization 



The transformations ([15| and (16) do not affect the 
stability condition ([6|. Taking into account that G = 



6l becomes J2p„ ^^pqWpq < 0: 
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Applying the first transformation (15) to (27 1 results 
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The last term in the left part vanishes {J2pq ^^pq 
preserving the form of condition (pi. The second trans 



formation (16) converts (28) into 



:^^Anp,w;'^<o. 



M. Milinski checked the readiness of a single stickle- 
back fish to inspect a predator (located in the front of a 
pool) in presence of permanently cooperative or defecting 
companion, which was simulated by a tilted mirror [T^. 
The "cooperating mirror" was parallel to the pool, sim- 
ulating a companion which followed immediately a pro- 
ceeding stickleback. The "defecting mirror" was tilted 
to simulate a companion which stayed increasingly be- 
hind a proceeding individual. The results which are 
relevant for this work are [19]: "With the cooperating 
mirror the sticklebacks twice as often in the front half 
(mean Pc — 25.1%) as with the defecting mirror (mean 
Ps — 12.1%)", where Pc and Pg are the probabilities 
to observe a stickleback close to the predator in the ex- 
periments with cooperative and defecting mirrors respec- 
tively. 

Associating predator inspection (going to the front half 
of the pool) with cooperative behavior and taking into 
account that in the case of a "cooperating mirror" only 
S vs. S or C vs. C interactions are observable, one ob- 
tains: 



cc 
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C. Stability analysis 

Evolutionary stability of {ast,Pst) (a solution of ([t])) 
can be justified by population dynamics analysis, if the 
corresponding condition ([8|) vanishes. The population 



The same applied to the case of "defecting mirror" 
(C vs. S or S vs. S interactions are observable) leads 
to: 
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where ^Ipq are the probabilities of the outcomes in a hy- 
pothetical experiment with two sticklebacks (rather than 
a stickleback and a mirror) possessing the same {a, (3). 

To derive conditional probabilities {a, (3) of a stickle- 
back and corresponding interaction statistics fipq, sub- 
stitution of Pc 0.251 and Ps = 0.121 to eqs. (|30|, 
( 131) and ([TT]) results in a = 0.121 and P = 

(T~- a) /{I + 2a) = 0.708. The corresponding values 
flpq following ([Tl| are flee = 0.207, flss = 0.621 and 
^SC = ^cs = 0.086. As one can see, neither flee nor 
flss are negligible in this case. 



to a fight f20j . To avoid risk of fighting "certain indi- 
viduals consistently lag behind their companions during 
the group response". The most surprising phenomenon 
is that "although leaders recognize laggards and behave 
more cautiously in their presence, they continue to lead 
the response" . The separation into selfish laggards and 
cooperative leaders " appears early in life and persists into 
adulthood" . 



E. Lions defending their territory 

Female lions defend the pride's territory by approach- 
ing a potential intruder and when necessary proceeding 



In this case the reported observations are insufficient 
to derive statistics of interaction outcomes flpq, where 
laggard and leader are associated with selfish and coop- 
erative behaviors respectively. 
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FIG. 1: Prom observed behavior to evolutionary payoffs and back. Animal interactions may be classified according to 
the behavior of the competitors, using observable parameters such as level of aggression or cooperation. Four types of contests 
exist when the individual behavior is either selfish S or cooperative C {S vs. S, S vs. C, C vs. S and C vs. C). Continuous 
observations of such competitions provide Qpq, the probabilities of p vs. q interactions {p,q G {S,C}). The statistics flpq 
defines individual behavior parameters actserv and dobscrv, the conditional probabilities to be cooperative against selfish and 
cooperative competitors respectively. The corresponding Darwinian payoffs may be determined by analysis of evolutionary 
stability of the observed behavior. When payoffs are known, one can predict evolutionarily stable behavior {acaic, dcaic) and 
compare it with the observed one {oobserv, l3observ)- Animals with documented statistics of interactions and payoffs, such as 
the bowl and doily spiders, provide an ultimate test for evolutionary theory. 
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FIG. 2: Darwinian payoffs and corresponding evolutionarily stable behavior. Darwinian payoffs b and c Q are 
generally presented as model games of Leader (fe > c, c > 1), Battle of the Sexes {b < c,b > 1), Chicken (6 > 1,0 < c < 1) 
and Prisoner's dilemma (6 > l,c < 0). For each values b and c the corresponding prediction of evolutionarily stable behavior 
{a, P) is indicated. Chicken and Leader games cause unconditional selfishness against cooperative competitors (/3 = 0, with 
corresponding Q.cc = 0). Battle of the Sexes leads to permanent cooperation against selfishness (a = 1, with corresponding 
^ss = 0). No predictions can be made for other cases, since stability analysis of the corresponding {a, (5) parameters requires 
previous knowledge of some difficult to observe interaction characteristics, such as information exchange, the order of mutual 
responses etc. 
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FIG. 3: Evolutionary stability of behavior (a,/3). Evolutionarily stable behavior (a, 0) has to outperform any possible 
mutant. In the {a, (3) space only interactions between individuals on the same boundary can be analyzed (,9 = with 
corresponding ficc = 0, a = 1 with ^Iss = 0, /? = 1 with ficc = 1, a = with Qss = !)■ Convergence of a population 
towards its stable position can be analyzed only along the boundaries (solid arrows). Mutants along the boundary will converge 

towards the stable point, while dynamics of mutants outside the boundary (dashed arrows) can not be generally analyzed. 
These mutants do not affect the position of the stable point on the boundary. 



